Rapid diagnostic tests versus RT–PCR for Ebola virus infections: a systematic review and meta-analysis

Abstract Objective To evaluate the clinical accuracy of rapid diagnostic tests for the detection of Ebola virus. Methods We searched MEDLINE®, Embase® and Web of Science for articles published between 1976 and October 2021 reporting on clinical studies assessing the performance of Ebola virus rapid diagnostic tests compared with reverse transcription polymerase chain reaction (RT–PCR). We assessed study quality using the QUADAS-2 criteria. To estimate the pooled sensitivity and specificity of these rapid diagnostic tests, we used a bivariate random-effects meta-analysis. Findings Our search identified 113 unique studies, of which nine met the inclusion criteria. The studies were conducted in the Democratic Republic of the Congo, Guinea, Liberia and Sierra Leone and they evaluated 12 rapid diagnostic tests. We included eight studies in the meta-analysis. The pooled sensitivity and specificity of the rapid tests were 86% (95% confidence interval, CI: 80–91) and 95% (95% CI: 91–97), respectively. However, pooled sensitivity decreased to 83% (95% CI: 77–88) after removing outliers. Pooled sensitivity increased to 90% (95% CI: 82–94) when analysis was restricted to studies using the RT–PCR from altona Diagnostics as gold standard. Pooled sensitivity increased to 99% (95% CI: 67–100) when the analysis was restricted to studies using whole or capillary blood specimens. Conclusion The included rapid diagnostic tests did not detect all the Ebola virus disease cases. While the sensitivity and specificity of these tests are moderate, they are still valuable tools, especially useful for triage and detecting Ebola virus in remote areas.


Introduction
Ebola virus disease was first discovered in 1976 in the Democratic Republic of the Congo and South Sudan. 1 This highly pathogenic disease is often fatal in humans; in past outbreaks the case fatality rate ranged from 25% (37/149) to 90% (128/143). 1 At the initial stages of the disease, symptoms include fever, vomiting, diarrhoea, anorexia and fatigue. 2 Diagnosing the disease on these symptoms alone is challenging, because they are similar to common endemic diseases present in Africa, such as typhoid fever, malaria and yellow fever. 3,4 To confirm the diagnosis of Ebola virus disease, a positive result from a reverse transcription polymerase chain reaction (RT-PCR) test is required. 5 However, lateral flow assays (that is, rapid diagnostic tests) are valuable tools to limit the spread of the disease since their fast turnaround time has the potential to trigger early outbreak alerts. For instance, researchers estimated that if a combination of rapid diagnostic tests and RT-PCR assays had been available during the 2013-2016 Ebola virus disease outbreak, the number of infections would have been up to a third less in Sierra Leone. 6 Most Ebola outbreaks begin in remote or rural areas 1 with limited hospital availability and trained clinicians. Laboratory equipment needed for diagnosis and trained equipment users are rarely available; it can take hours or days to get the RT-PCR results. 7 If rapid diagnostic tests for the disease were readily available in high-risk outbreak areas, lives could be saved since the time between virus introduction into a community and implementation of countermeasures could be decreased. 8 According to the World Health Organization (WHO), rapid diagnostic tests for Ebola virus should have a desired clinical sensitivity of > 98% and an acceptable clinical sensitivity of more than 95%. 9 Since 1976, many Ebola virus rapid diagnostic tests have been developed, but researchers have not yet thoroughly assessed the evidence of their performance in clinical samples. The few rapid diagnostic tests that have been assessed in field conditions demonstrated uncertainty and variability in performance. 6 We therefore conducted a meta-analysis to increase the evidence base of current rapid diagnostic tests detecting Ebola virus in suspected cases.

Methods
We conducted a systematic review and meta-analysis of studies that assessed the performance of rapid diagnostic tests for Ebola virus compared with RT-PCR. We followed the Preferred Reporting Items for a Systematic Review and Metaanalysis of Diagnostic Test Accuracy Studies. 10 This review is registered with the International Prospective Register of Systematic Reviews (CRD42021278280).
We searched MEDLINE®, Embase® and Web of Science for articles published from 1976 to 7 October 2021. Search terms used are available in Box 1. We applied no language restrictions during the search. We also hand-searched the articles included in the reference lists of relevant studies, related key reviews and a book chapter on Ebola rapid diagnostic tests. 8 The studies that we retrieved were exported to EndNote software X9 (Clarivate, Philadelphia, United States of America) and from there, we removed duplicated studies.
Objective To evaluate the clinical accuracy of rapid diagnostic tests for the detection of Ebola virus. Methods We searched MEDLINE®, Embase® and Web of Science for articles published between 1976 and October 2021 reporting on clinical studies assessing the performance of Ebola virus rapid diagnostic tests compared with reverse transcription polymerase chain reaction (RT-PCR). We assessed study quality using the QUADAS-2 criteria. To estimate the pooled sensitivity and specificity of these rapid diagnostic tests, we used a bivariate random-effects meta-analysis. Findings Our search identified 113 unique studies, of which nine met the inclusion criteria. The studies were conducted in the Democratic Republic of the Congo, Guinea, Liberia and Sierra Leone and they evaluated 12 rapid diagnostic tests. We included eight studies in the meta-analysis. The pooled sensitivity and specificity of the rapid tests were 86% (95% confidence interval, CI: 80-91) and 95% (95% CI: 91-97), respectively. However, pooled sensitivity decreased to 83% (95% CI: 77-88) after removing outliers. Pooled sensitivity increased to 90% (95% CI: 82-94) when analysis was restricted to studies using the RT-PCR from altona Diagnostics as gold standard. Pooled sensitivity increased to 99% (95% CI: 67-100) when the analysis was restricted to studies using whole or capillary blood specimens. Conclusion The included rapid diagnostic tests did not detect all the Ebola virus disease cases. While the sensitivity and specificity of these tests are moderate, they are still valuable tools, especially useful for triage and detecting Ebola virus in remote areas.
Two authors screened all the titles and abstracts identified through the search, and reviewed the full text of potentially relevant articles against the inclusion and exclusion criteria (Box 2). Each researcher was blind to the selection of the other researcher. We recorded reasons for excluding articles; disagree-ments were discussed and arbitrated by consensus.

Data extraction
Two investigators independently extracted data from individual studies. We then compared extracted data and any disagreements were resolved through discussion.
Prior to data extraction, we designed a standardized data extraction form. Extracted data included: setting, study period, sample size, type of specimen, index test, reference standard and reported conflicts of interest. We also extracted reported sensitivity and specificity, raw data on true positives, false positives, false negatives and true negatives to recreate two-by-two tables.

Methodological assessment
Two authors independently evaluated the methodological quality of the included studies using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool. 11 We resolved any disagreements through consensus and further article reading.

Data synthesis and analysis
All statistical analyses were performed using Stata (version 16, StataCorp LP, College Station, USA). We applied a bivariate-effect model to calculate the pooled sensitivity, specificity, positive and negative likelihood ratios and the diagnostic odds ratio (OR). We generated forest plots of the sensitivity and specificity of each data point.
We conducted meta-analysis using the bivariate-effect models because of the heterogeneity expected between studies assessing diagnostic test accuracy. 12 We also applied the bivariateeffect model to generate plot hierarchical summary receiver-operating characteristic curves.
Heterogeneity was assessed by visual inspection of forest plots of the sensitivity and specificity and the shape of the hierarchical summary receiveroperating characteristic curves. 13 In our subgroup analysis, we stratified studies into seven subgroups based on: specimen type (serum alone, plasma alone, serum and plasma combined, whole blood and capillary blood combined); reference standard ( We further narrowed searches by including the following names for index test: "ReEBOV, " "QuickNaviTM-Ebola, " "eZYSCREEN, " "DSTL EVD lateral flow assay, " "OraQuick Ebola rapid antigen test kit, " "SD Q Line Ebola Zaire Ag" and "NMRC EBOV LFI. " Box 2. Inclusion and exclusion criteria used to identify studies on diagnostic accuracy of rapid tests for Ebola virus disease PCR tests such as Trombley assay); 14 and exclusion of outliers (studies reporting that the rapid diagnostic test is 100% sensitive or specific). We also performed a subgroup analysis using only the ReEBOV™ Antigen Rapid Test kit (Corgenix Inc., Broomfield, USA; hereafter ReEBOV™) because it had enough data points to be pooled separately.

Index tests
The included studies evaluated 12 index tests: Reba™; 17,21,22 QuickNavi™-Ebola (Denka Seiken, Tokyo, Japan); 16 23 and three tests using the dual path platform from Chembio Diagnostics (Medford, USA): Fever Panel Antigen System, 15 Ebola Antigen System 15 and Ebola-Malaria Antigen duplex system. 15 Further details on the tests are available from the data repository. 34 The tests using the dual path platform are different from classic lateral flow assays as they contain a cartridge with a battery. ReEBOV™ should be stored at 2-8 °C, hence a cold chain is needed which could be a concern for use in remote field conditions. 17,22 In addition, ReEBOV™, DEDIATEST EBOLA and SD Ebola Zaire Ag have been reported to have operational biosafety concerns when using the test. 17 The NMRC Ebola virus lateral flow immunoassay could not yield readable results in samples containing red blood cells. 20 All studies compared the rapid diagnostic test against RT-PCR, notably altona, 15,[17][18][19][20]22,23 Trombley assay, 17,20 GeneXpert® Ebola (Cepheid, Sunnyvale, USA) 16 , Ebola-specific quantitative RT-PCR (while not mentioned specifically in the article, the authors cite the Trombley assay in their methods), 21 and Weidmann technique (that is, a quantitative one-step RT-PCR). 19

Methodological assessment
The results of the quality assessments are available in the data repository. 34 In the patient selection domain, 78% (7/9) of the studies had an unclear or high risk of bias, because these studies did not clearly specify random or consecutive recruitment of participants. 15,17,18,[20][21][22][23] Furthermore, some studies had suboptimal study design (two studies), 15,22 missing information on patients' exclusion criteria (seven studies) 15,17,18,20,21,22,23 and the use of stored blood samples collected for other purposes (three studies). 15,17,18 Regarding the reference standard domain, one study was judged as having a high risk of bias for incorrect use of altona's RT-PCR kit, by modifying the manufacturer's instructions. 22 We judged applicability concerns to be low in the patient selection, index test and reference standard domains.
Five (56%) studies explicitly stated no conflicts of interest. 15,16,19,20,23 In two (22%) studies, authors acknowledged having potential conflicts of interest. 18,21 Two (22%) studies reported receiving test kits from manufacturers but did not consider it as a conflict of interest. 17,22

Meta-analysis
Nineteen data points covering 5332 tests performed were available to summarize the performance of rapid diagnostic tests for Ebola virus. Table 2 shows the bivariate-effect model estimates for the pooled sensitivity, specificity, positive and negative likelihood ratios and the diagnostic OR. The pooled sensitivity and specificity were 86% (95% confidence interval, CI: 80-91) and 95% (95% CI: 91-97), respectively (Fig. 2). While sensitivity estimates varied widely from 62% to 100% across studies, the range for specificity estimates were narrower (80-100%; Fig. 3).

Subgroup analysis
In the subgroup analyses, the pooled specificity estimates were more consistent across subgroups compared with pooled subgroup sensitivity estimates ( Table 2).

Systematic reviews
Rapid diagnostic tests for Ebola virus infections Basilua Andre Muzembo et al.

Reference standard altona
We restricted the analysis to 13 data points (2925 specimens tested) where the altona RT-PCR kit had been used as the gold standard. 15

Discussion
In this study, we conducted metaanalyses on clinical accuracy studies to assess the performance of rapid diagnostic tests to detect Ebola virus in individuals suspected of having the disease. Compared with the gold standard RT-PCR, the overall pooled sensitivity for rapid tests was 86%, falling short of a desired sensitivity of > 98% and an acceptable sensitivity of > 95% listed in a WHO target product profile document released during the 2013-2016 Ebola virus disease outbreak. 9 Furthermore, we show that the overall pooled specificity was 95%, also lower than WHO's recommended level of > 99%. 9 While the clinical value for Ebola virus screening to contain the outbreak is indisputable, our findings suggest that better performing rapid diagnostic tests in field conditions are needed. Our results indicate that current tests miss 14% of cases, which is a considerable proportion because of the contagiousness and high mortality of Ebola virus disease. False-negative results should be minimized to the lowest level possible, since false-negative individuals might infect other people. In hospitals, false-negative patients might be treated with less precaution than positive patients and hence the likelihood of infecting health-care workers and other patients is greater. Allowing false-negative patients to wait at home for a confirmatory RT-PCR test increases the risk of infecting people in the community. In addition, false-negative patients will not be included in contact tracing, which might lead to the transmission chain being sustained. Of note is that a low viral load in the specimen could also lead to a false-negative result, even with a rapid test with high sensitivity. 36 False positivity can have severe implications for false-positive individuals and their families since they would be subject to unnecessary quarantines. These individuals can also be exposed to the potential Ebola patients when waiting for the confirmation of the diagnosis.
When assessing in which type of sample the rapid test performed best, we found that tests made on whole or capillary blood had the highest sensitivity, specificity, likelihood ratios and diagnostic OR. Using whole or capillary blood has the advantage that blood centrifugation is not required, which would reduce the turnaround time and make the test more accessible in remote field settings. However, this subgroup analysis included only five data points, from four studies, 16,19,22,23 and two of the data points were from a study 22 that has been a subject of debate. [37][38][39][40] Hence, further field studies are required to confirm results.
Using a different gold standard also affected the results. The six data points with higher sensitivity all used altona as the gold standard. 15,17,22,23 We also noted that pooled sensitivity was higher in studies using altona (90%) compared with other gold standards (78%). However, only six data points were available for other gold standards versus 13 for altona. It is unclear why using altona yielded a sensitivity superior to that of other gold standards. Reduced sensitivity of altona to specimens with cycle threshold values above 30 (i.e. low viral loads) have been observed. 5,41 This reduced sensitivity of altona may affect the interpretation of our pooled estimates. Our pooled sensitivity might have been underestimated, since altona may fail to detect positive samples with low viral loads. However, we cannot rule out the possibility that studies using altona might also overestimate the sensitivity of rapid tests, since one study that used altona reported an unusual sensitivity of 100%. 22 The review identified sufficient data points for assessing ReEBOV™ performance. This test received an emergency use authorization from the United States Food and Drug Administration (FDA) and WHO during the 2013-2016 Ebola outbreak. 26 But, in 2018, FDA revoked this authorization when the new manufacturer (Zalgen Laboratories) that had acquired the company failed to reproduce the claimed test accuracy of ReEBOV™. 42 The claimed sensitivity and specificity of the test were 91.8% and 84.6%, respectively. 6 Our results are in line with the suboptimal performance: we showed a pooled sensitivity of 95% with a wide confidence interval (95% CI: 70-99), suggesting that the result is overestimated due to included outlier studies. 19,22,23 The pooled specificity was also below WHO criteria of 99% analytical specificity. 9 This study has some limitations. First, our meta-analysis included some studies with some methodological limitations and studies where tests' accuracy could have been overestimated. However, this concern was addressed by conducting subgroup analyses. For instance, we assessed whether the pooled estimates differed by removing outliers in the meta-analysis. This approach did indeed change the pooled sensitivity but not its specificity. Second, synthesizing the included studies in one pooled estimate of sensitivity and specificity could be inaccurate as there was substantial variation in the used cut-off for RT-PCR cycle threshold (data repository), 34 which limits comparability of rapid tests. These issues are illustrated by pooling only data from ReEBOV™ studies. The results showed a suboptimum pooled sensitivity with a great uncertainty.
Third, some studies used stored blood samples. Using frozen blood samples that were processed through freeze-thaw cycles may have compromised the tests' sensitivity. Nevertheless, this explanation is unlikely since a newly published study conducted on patients also demonstrated that the sensitivity of three rapid tests (QuickNavi-Ebola, OraQuick Ebola Rapid Antigen Test (OraSure Technologies, Bethlehem, USA) and EBOLA Ag K-SET rapid test) varied, from 40% to 87%. 43 Thus, for Ebola rapid tests with potential acceptable sensitivity, well designed clinical studies with larger sample sizes are necessary for an adequate assessment of their current performance.
Finally, we could not perform subgroup analysis comparing test performance by symptoms duration because of lack of data stratified by timing of the tests. Therefore, future field studies need to evaluate the performance of these tests in relation to symptoms duration. As an example, rapid diagnostic tests for severe acute respiratory syndrome coronavirus 2 are more sensitive (83.8%) when used within 7 days of symptoms onset than when used at later than 7 days (sensitivity of 61.5%). 44 These limitations, however, would not modify the usefulness of Ebola rapid diagnostic  The strength of our study is that our literature was extensive, without any language restrictions, although only studies published in English and French were included. However, some studies were excluded because of low evidence on clinical performance, and studies could have been missed by our search strategy.
In conclusion, the results from this meta-analysis suggest that currently there is no commercial rapid diagnostic test for Ebola virus disease that has sufficient sensitivity and specificity to meet WHO standards. Despite the suboptimal performance, these tests still have clinical value because of their rapid turnaround time. Clinicians, especially those in settings where RT-PCR tests are not immediately available, should be aware of the existence, availability and limitations of the rapid tests. A negative result is unreliable in a subject highly suspected of having Ebola virus disease and the result must be confirmed using RT-PCR. Current commercially available tests include ReEBOV™, SD Zaire Ag, OraQuick, Ebola Antigen System and QuickNavi-Ebola, 45 and health-care workers can procure rapid tests, such as OraQuick, from the US Centers for Disease Control and Prevention or WHO. 46